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In Ihe Fall 1991 semesier. the Measurement and F\alualion Cenler (MFC) was asked by Ihe 
ccK)rdinalor of louer division Japanese courses in ihe Department of Oriental and African 
Languages and Literatures (OAL) to help develop a test for credit by examination in the four ]o\\cr 
division Japanese courses (Japanese 506, Japanese 507. Japanese 4I2K. and Japanese 412L) 
offered at the L^niversity of Texas at Austin (UT Austin). All test items were written by. instructors 
of Japanese and reviewed by statT members from the MFC. The study tcx)k place between the Fall 
1991 and Spring 1993 semesters, inclusive. 

Method 

Materials 

hi consultation with MFC staff members, the UT Austin Test for Credit by Examination in 
Japanese was constructed from locally developed items provided by instructors of Japanese in the 
OAL Depanment. llie test consists of eighty items distributed among three sections: grammar, 
listening, and reading. Forty of the items test grammar skills, twenty test listening skills, and the 
remaining twenty test reading skills. The item difficulty levels were evenly divided between the 
four lower division Japanese courses. 

The listening section of the test was recorded on tape by the instructors in modem standard 
Japanese and consists of two parts. The first part contains eight items. The examinee listens to a 
statement in Japanese and is asked to choose the most appropriate response (in English). For part 
two of the listening section, there are twelve questions based on a short conversation. All forty of 
the items in the grammar section require sentence completion. Finally, the reading section contains 
four passages. For each passage the examinee responds to five questions. 

The test was initially administered to all students enrolled in Japanese 506 and Japanese 412K 
in the Fall of 1991, There were 166 students in the former group and 66 from the latter. Based on 
the results of an item analysis, test items with poor psychometric properties were edited or 
removed and new items were introduced. The revised test was administered to all students 



rcgisicrcd for Japanese 507 and 4121. during the Spring 1992 semester and to all students 
registered for Japanese 506 and 412K during the ^"all 1992 semester. To insure adequate 
participation, the test accounted for 8% of each student's final grade in the course. All tests were 
administered during the tlnal examination period. 

Subjects 

Test \alidation began in the }-all of 1991. During that semester the test was administered to all 
students enrolled in Japanese 506 and Japanese 412K. Based on item analysis, the test was 
revised and readministered during the Spring 1992 and Fall 1992 semesters. 

In the Spring 1992 semester, the test was administered to 170 students. One hunH.red twelve 
(112) of the 170 students \\ere enrolled in Japanese 507 (second semester Japanese), while the 
remaining students (58) were enrolled in Japanese 4121. (founh semester Japanese). In the f-all 
1992 semester, 231 students were tested. One hundred thirty-nine (139) of the 231 students were 
enrolled in Japanese 506 (second semester Japanese), while the remaining students (92) were 
enrolled in Japanese 41 2K i )urth semester Japanese). The above numbers include students with 
both a preliminary final grade and a test score. Individuals who failed to take the test or who did 
not receive a preliminary final gnide were excluded from the study. 

Procedure 

Following administration of the test at the end of each of the three semesters, the test results 
were collected along with tJie preliminary course grades. The preliminary course grades were 
submitted by the instructors as an evaluation of each student's performance on all parts of the 
required course work except the UT Austin test portion of the final examination grade. In most 
inst/inces, the faculty members supplied the preliminary grades on tu'o different scales: a 
numerical percentage-grade scale of 0-100 and the traditional five point letter-grade scale of A 
through F. Alt of the instructors used the traditional cut points of 90, 80, 70 and 65 for the 
assignment of letter grades. In cases where letter grades were not provided, MHC's data 
processing division converted the numerical percentage-grade to the letter-grade scale. 

Several analyses were then performed. An item analysis was done to determine if there were 
an\ items that required re\ ision or removal. Because of the number of such items at the end of the 
first semester of the study, the test was revised and administered the next semester. Results from 
the other item analyses (i.e., from the second and third semesters of the study) showed that further 



test rev ision was not needed. These results indicated that the test scores would be appropriate for 
the analyses required for the setting of cut scores. 

Analyses to determine the relationship of student performance on the test, as maisured by the 
Total Raw Score, to student performance in the course, as measured by the f^eliminary Course 
Grade, were then performed. Crosstabulations of test scores and preliminary grades on a tne 
point A through F scale, by course, with mean scores and standard deviations tor students with 
each letter grade and tor all students combined were prcxluced. The frequency distributions of lest 
scores were then collapsed trom the five point scale into tv\o groups denoting unsatisfactory 
performance in the course (Preliminary Course Grades F and D) and satisfactory course 
performance (Preliminary Course Grades C B and /\). 

Using the descriptive statistics obtained for each course, MEC staff members also calculated 
the coefficient of correlation between Total Raw Score on the placement test and Preliminar>' 
Course Grades. Regression equations were derived from the data in order to predict test scores 
(Expected Scores) from Preliminary Course Grades on the A through F scale, and final grades 
(Expected Grades) in the course from the Total Raw Score on the raw score scale of 0-80, 

In addition, staff members of the MEC determined the accuracy of placement decisions that 
would have occurred at each score interval had students who participated in the study actually been 
placed in courses according to the scores they made on the test. Accurate placement was labeled as 
Correct placement: inaccurate placement was classified in tenns of two placement errors -- 
placement Too High and placement Too Low. 

Basing their choices on the results of all of the above analyses, MEC sta^f members then 
selected for each course a series of recommended decision ("cutting") scores to present to the 
department. In their selection, they made use of a series of six guidelines often recommended for 
use by various authorities. Each guideline refers to a specific value obtained in the several analyses 
of the data collected in the study. 

Finally, MEC staff memlK^rs met with the instaictors and reported the results of the testing 
project, and the instructors submitted sectional grade rosters on which they had recorded a 
Preliminary' Course Grade (defined above) for each student. Four pieces of information included 
in the test results repc^rted to the instaictors were alphabetical scx:tion rosters with Listening, 
Grammar, Reading, and Total Raw Scores and the percentage correct equivalents of the numerical 
raw scores for each student: similar rosters but with the students' names arranged according to the 



4 



descendip.g order of Iheir Total Raw Scores: trcquency distributions of the four raw scores, with 
mean (average) scores and standard deviations (measures of the variability among the scores) for 
each of the four raw scores: and a breakdown of the number of items answered correctly by each 
student. MI:C staff members also demonstrated how the Total Raw Score could be incor[X)rated 
into the students' tlnal grade. 

Results 

The data are presented in tour sets of four tables, luwh set corrcsfx^nds lo one of the four 
lower division Japanese courses in this study. Tables 1.1 through 1.4 are based on information 
from Japanese 506: Tables 2. 1 through 2.4 are based on Japanese 507: Tables 3. 1 through 3.4. on 
Japanese 41 2K; and Tables 4J through 4.4, on Japanese 41 2L. The tlrst table in each set contains 
a crosstabulation of test scores by preliminary course grades. In addition, expected grade for each 
test score and expected test score tor each preliminary course grade are calculated tor each table 
based on the regression equations shown in the bottom right hand side of the tables. These 
equations are calculated by using data solely from the course under examination. Expected grades 
greater than 4.00 are recorded as 4,00. Summary statistics, mean test scores and their standard 
deviations are provided for each preliminary grade, as well. The second table in each set contains 
the same crosstabulations as described above except the preliminary course grades are collapsed 
into two performance categories, unsatisfactory and satisfactory, using the criteria noted earlier. 
The third table in each set presents information on the placement accuracy for a range of test 
scores. Each test score is examined as a potential cutoff point for credit in the course under 
examination. Given tne information in the previous table and for each score, it is determined how- 
many individuals from the unsatisfactory group would receive credit (placed too high), how many 
from the satisfactory group would not receive credit (placed too low), and how many would be 
correctly placed from each of the two groups. This information is presented as frequencies and 
percentages examining correct placement for the two groups separately and together (overall 
accuracy). The final table in each set gives the scores suggested by six guidelines for use in 
selecting decision scores. 

Discussion 

Several obser\'ations can be made about the data. I-irst. the correlation coelYicients for all four 
sections are accepaible. In pre\ious studies correlations have generally fallen w ithin the range of 
0,40 lo OXiO, The two lowest correlations. 0.43 tor Japanese 506 and 0,47 for Japanese 507. 
which are at the low end of the range given above» may be due to restriction of range. In both 
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Table 1.1 

End-of-Course Scores on The UT Austin Test for Credit in Japanese in 
Relation to Perlormance in Japanese 506: Frequency Distributfons. 
Descriptive Statistics. Regression Eauations. Expected Grades, 
and Expected Scores-Fall. 1992 
(N = 139) 



Final Gfaaes in jaoanaese 506 



Test 


Expected 


0 


1 


2 


3 


4 


Total 


Scores 


Grades 


F 


D 


C 


8 


A 


N 


49-73 


4.00 




1 111 


4 ! 


5 


48 


3.97 




\ \ i 


1 0 


47 


3.91 




\ \ ^ 




0 


46 


3.84 




1 i 


1 


1 


45 


3.78 




1 1 I 




0 


44 


3,72 




1 111 


1 


2 


43 


3.65 






\ 1 3 


3 


42 


3.59 






1 1 


1 1 


3 


5 


41 


3.52 






i 1 3 


3 


40 


3.46 








1 


2 


3 


39 


3,39 








1 


5 


6 


38 


3.33 






1 


3 


8 


12 


37 


3.26 








1 


1 


2 


36 


3.20 






2 


1 


4 


7 


35 


3.14 






4 


1 


6 


11 


34 


3.07 








6 


10 


16 


33 


3.01 




1 


2 


6 


4 


13 


32 


2.94 






3 


2 


1 


6 


31 


2.88 




2 


4 


4 




10 


30 


2.81 




1 


2 


4 


1 


8 


29 


2.75 




1 


1 


3 


1 


6 


28 


2.69 




1 


4 


1 


2 


8 


27 


2.62 


1 




1 


2 


1 


5 


26 


2.56 


1 


1 


1 


1 




4 


25 


1 2.49 












0 


24 


' 2.43 








1 




1 


23 


2.36 








1 


0 


22 


2.30 












0 


21 


2.24 












0 


20 


2.17 












0 


19 


2.11 










1 


1 


18 


2.04 












0 


17 


1.98 












0 


16 


1.91 






1 






1 


Total 


2 


7 


27 


41 


62 


139 


% 


1% 


5% 


19% 


30% 


45% 


100% 


Mean Score 


26.5 


29.7 


31.4 


33.3 


37.6 


34.6 


Standard Deviation 


0.50 


2:.12 


4 71 


4.89 


7.23 


6.59 


Expected Score 


26 


28 


31 


34 


37 





Expected Grade = 

(Test Score x 
0.06433) + 0.8845 



Expected Score = 
(Preliminary Grade x 
2.9133) + 25.5068 



Mean 


Standard 


Grade 


Deviation 


3 11 


0.98 



Coeflicient of 
Correlation 
r« 4329 
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Table 1 2 

End -of -Course Scores on The UPAustin Test tor Credrt in Japanese in 
Relation to Performance in Japanese 506: Combined Frequency 
Distributions and Descnptive Statistics 
Fall, 1992 
(N = 139) 





Final Graaes m Japanese 506 1 


Test 


Unsatisfactory 


Satisfactory 


Total 


Scores 


fO. 1) 


f2-4^ 


N 


49-/3 




5 


5 


48 






0 


4/ 






0 


A C 

4b 




1 


1 


4d 






0 


44 




2 


2 


4o 




3 


3 


A O 

42 




5 


5 


A ^ 

4 1 




3 


3 


A A 

40 




3 


3 


on 

oy 




6 


6 


3o 




12 


12 


J / 




2 


2 


OD 




7 


7 


35 




11 


11 


34 




16 


16 


oo 

33 


1 


12 


13 


32 




6 


6 


31 


2 


8 


10 


30 


1 


7 


8 


29 


1 


5 


6 


28 


1 


7 


8 


27 


1 


4 


5 


oo 


2 


2 


4 


25 






0 


24 




1 


1 


oo 
<i3 






0 


22 






0 


21 






0 








0 


19 




1 


1 


18 






0 


17 






0 


16 




1 


1 


Total 


9 


130 


139 


% 


7% 


94% 


100% 


Mean Score 


290 


34 9 


34 6 


Standard Deviation 


2.31 


6.62 


6.59 



Mean 


Standard 


Grade 


Deviation 


3.11 


0.98 



CoeHicient of 
Correlation 
r= 4329 
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Table 1 .3 

Scores on !ne UT Aus'.in Test tor Credit in Japanese in Relation to 
Student Perfo'mancs in Japanese 506: Possible Decision Scores 
and Corresponding Accuracies of Placerrent-Fail, 1992 

(N« 139) 



Place- 


Cumulative Nunrsoer 
of Students 


Percent of Students in 
Each Placement Category 


Overall Accuracy 
of Placement 


ment 

Category 


Unsatisfactory 
0-1 (N^9) 


Satisfactory i 
2-4 iN« 130)1 


Ur:satisfactorv 


Satisfactory 


Placement 
Accuracy 


Number of 
Students 


%of 
Students 


35 - UD 
Below 35 


Too High 0 
Correct 9 


1 
1 

60 Correct j 
70 Too Low 1 


Tco H'gh 0% 
Correct 100% 


46% Correct 
54% Too Lov/ 


Too High 
Correct 
Too Low 


0 
69 
70 


0% 
50% 
50% 


34 - up 
Below 34 


Too High 0 
Correct 9 


76 Correct 
54 Too Low 


Too High 0% 
Correct 100% 


58% Correa 
42% Too Low 


Too High 
Correct 
Too Low 


• C 

85 
54 


0% 
61% 
39% 


33 - up 
Below 33 


Too High 1 
Correct 8 


88 Correct 
42 Too Lew 


Tco High 11% 
Correct 89%l 


55% Correct 
32% Too Low 


Too High 
Correct 
Too Low 


1 

96 
42 


1% 
69% 
30% 


32 - up 
Below 32 


Too High 1 
Correct 8 


94 Correct 
36 Too Low 


Too High 11% 
Correct 89% 


72% Correct 
28% Too Low 


Too High 
Correct 
Too Low 


1 

102 
36 


1% 
73% 
26% 


31 - up 
Below 31 


Tco High 3 
Correct 6 


102 Correct 
28 Too Low 


Too High 3?% 
Correct 67% 


78% Correct 
22% Too Low 


Too High 
Correct 
Too Low 


3 

108 
28 


2% 
78% 
20% 


30 • up 
t^"elow 30 


Too High 4 
Correct 5 


109 Correct 
21 Too Low 


Too High 44% 
Correct 56% 


84% Correct 
16% Too Low 


Too High 
Correct 
Too Low 


4 

114 
21 


3% 
82% 
15% 


29 - up 
Below 29 


Too High 5 
Correct 4 


114 Correct 
16 Too Low 


Too High 56% 
Correct 44% 


88% Correct 
12% Too Low 


Too High 
Correct 
Too Low 


5 

118 
16 


4% 
85% 
12% 


28 - up 
Below 28 


Too High 6 
Correct 3 


121 Correct 
9 Too Low 


Too High 67% 
Correct 33% 


93% Correct 
7% Too Low 


Too High 
Correct 
Too Low 


6 

124 
9 


4% 
89% 
6% 


27 - up 
Below 27 


Too High 7 
Correct 2 


125 Correct 
5 Too Low 


Too High 78% 
Correct 22% 


96% Correct 
4% Too Low 


Too High 
Correct 
Too Low 


7 

127 
5 


5% 
91% 
4% 


26 - up 


Too High 9 


127 Correct 


Tco High 100% 


98% Correct 


Too High 


9 


6% 


Below 26 


Correct 0 


3 Too Low 


Correct 0% 


2% Too Low 


Correct 
Too Low 


127 
3 


91% 
2% 


24 - up 

Below 24 


Too High 9 
Correct 0 


128 Correct 
2 Too Low 


Too High 100% 
Correct 0% 


98% Correct 
2% Too Low 


Too High 
Correct 
Too Low 


9 

128 
2 


6% 
92% 
1% 


19 ■ up 
Below 19 


Too High 9 
Correct 0 


129 Correct 
1 Too Low 


TooH:gh 100^/ 
Correct 0% 


99% Correct 
1% Too Low 


Too High 
Correct 
Too Low 


9 

129 
1 


6% 
93% 
1% 


16 - up 
Below 16 


Too High 9 
Correct 0 


130 Correct 
0 Too Low 


Too High 100% 
Correct 0^/ 


) 100% Correct 
5 0% Too Low 


Too High 
Correct 
Too Low 


9 

130 
0 


6% 
94% 
1 0% 
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Tabie 1.4 

Scores on The UT Austin Test for Credit In Japanese in Relation 
to Student Performance in Japanese 506: Scores Suggested 
by Six Guldelines.for Use in Selecting Decision Scores 
Fall. 1992 
(N = 139) 



Guideline Score 



1 . Expected Score for students whose performance in course was just minimally 
satisfactory (i.e., students with preliminary grades of C; see Expected Score 

row at bottom of Table 1 .1 ). 31 

2. Score for which Expected Grade was just minimally satisfactory (i.e.. C; see 1 8 
Expected Grade column in Tabie 1.1). 

3. Score for which percents of errors of students in each academic performance 
category (Unsatisfactory, Satisfactory) were rriost nearly equal. (See % Too High 

and % Too Low values in middle columns of Table 1 .3.) 31 

4. Score for which overall percents of errors were most nearly equal. (See % Too 

High and % Too Low values in last column of Table 1 .3.) 27 

5. Score that would have cut off (or held back) approximately the same number of 
students as were in the Unsatisfactory performance group. (See Tabie 1 .2 for the 
number of students in the Unsatisfactory group and the test score that most 

nearly Identifies that number of low-scoring students.) 27 

6. Score that would have maximized overall accuracy of placement. (See number 

Correct in next-to-last column of Table 1 .3.) 1 6 
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Table 2.1 

End-o(-Course Scores on The UT A jstin Test for Credrt in Japanese in Relation 
Japanese 507: Frequency Distributions, Descnptive Statistics. Regressio 
Expected Grades, and Expected Scores-Spring, 1992 
(U = 112) 



to Performance in 
n Equations. 



Final Graces m Japanese 507 



Test 


Expeaed 


0 


1 


2 


3 


4 


Total 


Scores 


Grades 


F 


D 


C 


B 


A 


N 


57-72 


4.00 




1 

1 






1 


1 


56 


3.94 




I 






1 


1 


55 


3.88 




1 






2 


2 


54 


3.81 






1 


1 


2 


53 


3.75 










1 


1 


52 


3.69 




1 




4 


4 


51 


3.63 




\ r 




2 


2 


50 


3.57 








1 


4 


5 


49 


3.51 






1 




1 


2 


48 


3.45 










3 


3 


47 


3.38 


1 






2 


6 


9 


46 


3.32 






1 




1 


2 


45 


3.26 




1 




3 


2 


6 


44 


3.20 






1 


3 


3 


7 


43 


3.14 




1 


2 




4 


7 


42 


3.08 








1 


1 


2 


41 


3.01 






1 


3 


1 


5 


40 


2.95 








1 


2 


3 


39 


2.89 








2 


2 


4 


38 


2.83 




1 


2 


6 


1 


10 


37 


2.77 






1 . 


2 




3 


36 


2.71 






1 


3 


1 


5 


35 


2.65 


1 




2 






3 


34 


2.58 






1 


1 


1 


3 


33 


2.52 


1 


1 


1 


1 




4 


32 


2.46 






1 






1 


31 


2.40 




1 






2 


3 


30 


2.34 








2 




2 


29 


2.28 


1 






1 




2 


28 


2.22 












0 


27 


2.15 


1 




2 


1 




4 


26 


2.09 






1 






1 


25 


2.03 












0 


24 


1.97 






1 






1 


23 


1.91 












0 


22 


1.85 








1 




1 


21 


1.78 








1 




1 




otal 


5 


5 


19 


36 


47 


112 


% 


5% 


5% 


17% 


32% 


42% 


100% 


Mean Score 


34.2 


38.0 


36.2 


38.4 


46.4 


41.2 


Standard Deviation 


7.00 


5.44 


6.89 


7 05 


7 07 


8,32 


Expected Score 


30 


34 


37 


41 


45 





Expected Grade = 

(Test Score x 
0.06151) + 0.4929 



Expected Score = 
(Preliminary Grade x 
3.639) + 30.1821 



Mean 


Standard 


Grade 


Deviation 


3.03 


1.08 



Coefficient of 
Correlation 
r =. .4731 
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Table 2.2 



End-of-Course Scores on The UT Austin Test for Credit in Japanese in 
Relation to Performance in Japanese 507: Combined Frequency 
Distnbutions and Descriptive Statistics-Spnng. 1992 
'N = 112) 





Final Graces in Jaoanese 507 




1 est 


Unsatisfactory 


Satisfactory 


Total 


Scores 


(0. ^) 




N 


57-/2 


I 1 


1 


56 


1 


1 


1 


55 


1 


2 


2 


54 


1 


2 


2 


53 


1 


1 


1 


52 


1 


4 


4 


51 


1 


2 


2 


50 


i 


5 


5 


49 


1 

1 


2 


2 




1 


3 


3 


47 


1 


8 


9 


46 




2 


2 


A C 

45 


1 


5 


6 


44 




7 


7 


4o 


1 


6 


7 






2 


2 


41 




5 


5 


Ar\ 




3 


3 






4 


4 


Jo 


1 


9 


10 


T7 




^ 


3 






5 


5 


oc 
x3d 


1 


2 


3 


x34 




3 


3 




2 


2 


4 






1 


1 


31 


1 


2 


3 


30 




2 


2 




1 


1 


2 


28 






0 


27 


1 


3 


4 






1 


1 


25 






0 


24 




i 

1 


1 








0 


22 




1 


1 


21 




1 1 


1 


Total 


10 


102 


112 


0/ 
/Q 


97o 


91% 


100% 


Mean Score 


36.1 


41 7 


41.2 


Standard Deviation 


6.55 


8.30 


8.32 



ERIC 



13 



Mean 


Standard 


Grade 


Deviation 


3.03 


1.08 



Coefficient of 
Correlation 
r - .4731 



11 



Table 2.3 

Scores on the UT Austin Test for Credit in Japanese in Relation to 
Student Performance m Jaoanese 507: Possible Decision Scores 
and Corresponding Accuracies of Placement-Soring. 1992 
fN - 112^ 





Cumulative Numoer 


Percent of Stuaents in 


Overall Accuracy 


Place- 




of Students 


Each Placement Category 




of Placement 


ment 


Unsatisfactory 


Satisfacto7 | 








Placement 


Number of 


%of 


Category 


0-1 


- 10^ 


2-4 


« 102)1 


Unsatisfactofv 


Satisfactorv 


Accuracy 


Students 


Students 


41 - up 


Too High 


3 


58 


Correct 


Too 'High 


30% 


57% Correct 


Too High 


3 


3% 


















Correct 


65 


58% 


Below 41 


Correct 


7 


44 


Too Low 


Correct 


70% 


43% Too Low 


Too Low 


44 


39% 


40 - up 


Too High 


3 


61 


Correct 


Too High 


30% 


60% Correct 


Too High 


3 


3% 


















Correct 


68 


61% 


Beiow40 


Correct 


7 


41 


Too Low 


Correct 


70% 


40% Too Low 


Too Low 


41 


37% 


39 " up 


Too High 


3 


65 


Correct 


Too High 


30% 


64% Correct 


Too High 


3 


3% 


















Correct 


72 


64% 


Below 39 


Correct 


7 


37 


Too Low 


Correct 


70% 


36% Too Low 


Too Low 


37 


33% 


38 - up 


Too High 


4 


74 


Correct 


Too High 


40% 


73% Correct 


Too High 


4 


4% 


















Correct 


80 


71% 


Below 38 


Correct 


6 


28 


Too Low 


Correct 


60% 


27% Too Low 


Too Low 


28 


25% 


37 - up 


Too High 


4 


77 


Correct 


Too High 


40% 


75% Correct 


Too High 


4 


4% 


















Correct 


83 


74% 


Below 37 


Correct 


6 


25 


Too Low 


Correct 


60% 


25% Too Low 


Too Low 


25 


22% 


36 - up 


Too High 


4 


82 


Correct 


Too High 


40% 


80% Correct 


Too High 


4 


4% 


















Correct 


88 


79% 


Below 36 


Correct 


6 


20 


Too Low 


Correct 


60% 


20% Too Low 


Too Low 


20 


18% 


35 - up 


Too High 


5 


84 


Correct 


Too High 


50% 


82% Correct 


Too High 


5 


4% 


















Correct 


89 


79% 


Below 35 


Correct 


5 


18 


Too Low 


Correct 


50% 


13% Too Low 


Too Low 


18 


16% 


34 - up 


Too High 


5 


87 


Correct 


Too High 


50% 


85% Correct 


Too High 


5 


4% 


















Correct 


92 


82% 


Below 34 


Correct 


5 


15 


Too Low 


Correct 


50% 


1 5% Too Low 


Too Low 


15 


13% 


33 - up 


Too High 


7 


89 


Correct 


Too High 


70% 


87% Correct 


Too High 


7 


6% 


















Correct 


92 


82% 


Below 33 


Correct 


3 


13 


Too Low 


Correct 


30% 


13% Too Low 


Too Low 


13 


12% 


32 - up 


Too High 


7 


90 


Correct 


Too High 


70% 


88% Correct 


T t J' u 

Too High 


7 


6% 


















Correct 


93 


83% 


Below 32 


Correct 


3 


12 


Too Low 


Correct 


30% 


12% Too Low 


Too Low 


12 


11% 


29 - up 


Too High 


9 


95 


Correct 


Too High 


90% 


93% Correct 


Too High 


9 


8% 


















Correct 


96 


86% 


Below 29 


Correct 


1 


7 


Too Low 


Correct 


10^/c 


) 7% Too Low 


Too Low 


7 


6% 


26 - up 


Too High 


10 


99 


Correct 


Too Hign 


100^/^ 


) 97% Correct 


Too High 


10 


9% 


















Correct 


99 


88% 


Below 26 


Correct 


0 


3 


Too Low 


Correct 


0^/ 


> 3% Too Low 


1 Too Low 


3 


3% 



23 • up 
Below 23 



Too High 10 
Correct 0 



1 00 Correct 
2 Too Low 



Too High 
Correct 



100% 
0% 



98% Correct 
2% Too Low 



Too High 
Correct 
Too Low 



10 
100 
2 



12 



Table 2.4 

Scores on The UT Austin Test for Credit In Japanese in Relation 
to Student Perlormance in Japanese 507: Scores Suggested 
by Six Guidelines for Use in Seleuing Decision Scores 
Spnng, 1992 
(N = 112) 



Guideline score 



1 . Expected Score for students whose perlormance in course was just minimally 
satisfactory (i.e., students with preliminary grades of C; see Expected Score 

row at bottom of Table 2.1). gy 

2. Score for which Expected Grade was just minimally satisfactory (i.e., C; see 25 
Expected Grade column in Table 2.1). 

3. Score for which percents of errors of students in each academic perlormance 
category (Unsatisfactory. Satisfactory) were most nearly equal. (See % Too High 

and % Too Low values in middle columns of Table 2.3.) 39 

4. Score for which overall percents of errors were most nearly equal. (See % Too 

High and % Too Low values in last column of Table 2.3.) 30 

5. Score that would have cut off (or held back) approximately the same number of 
students as were in the Unsatisfactory performance group. (See Table 2.2 for the 
number of students in the Unsatisfactory group and the test score that most 

nearly identifies that number of low-scoring students.) 30 

6. Score that would have maximized overall accuracy of placement. (See number 

Connect in next-to-last column of Table 2.3.) 2 1 



ERLC 
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Table 3.1 

End'Of-Course Scores on The UT Austin Test for Credit in Japanese in 
Relation to Performance in Japanese 41 2K: Frequency Distributions, 
Descriptive Statistics, Regression Equations. Expected Grades, 
and Expeaed Scores--Fall, 1992 
(N = 92) 



Final Grades m Jaoanaese 41 2K 



Test 


Expected 


0 


1 


2 


3 


4 


Total 


Scores 


Grades 


F 


D 


c 


B 


A 


N 


66-74 


4.00 








2 


5 


7 


65 


3.96 






1 






1 


64 


3.87 










1 


1 


63 


3.78 








1 


1 


2 


62 


3.69 








2 


1 


3 


61 


3.59 












0 


60 


3.50 






1 




2 


3 


59 


3.41 








1 


1 


2 


58 


3.31 










2 


2 


57 


3.22 






1 


1 


2 


4 


56 


3.13 








2 


2 


4 


55 


3.04 










2 


2 


54 


2.94 






1 


1 


2 


4 


53 


2.85 








1 


3 


4 


52 


2.76 




2 






2 


4 


51 


2.67 






1 


1 


4 


6 


50 


2.57 






2 


1 




3 


49 


2.48 




2 


1 


2 


1 


6 


48 


2.39 












0 


47 


2.30 




1 




3 




4 


46 


2.20 






1 


2 




3 


45 


2.11 






2 


1 




3 


44 


2.02 


1 


1 




1 




3 


43 


1.93 








3 




3 


42 


1.83 








1 




1 


41 


1.74 


H 

I 




1 






2 


40-33 


1.00-1.65 


8 


4 


1 


2 




15 


Total 


10 


10 


13 


28 


31 


92 


% 


11% 


11% 


14% 


30% 


34% 


100% 


Mean Score 


37.4 


43.7 


50.1 


50.9 


57.8 


50.8 


Standard Deviation 


3.35 


6.87 


7.25 


8.36 


6.45 


9.51 


Expected Score 


38 


43 


48 


52 


57 





Expected Grade = 

(Test Score x 
0.0926) - 2.0561 



Expected Score = 
(Preliminary Grade X 
4.7347) + 38.2907 



Mean 


Standard 


Grade 


Deviation 


2.65 


1.33 



Coefficient of 
Correlation 
r = .6621 



c 



i6 
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Table 3.2 

End-of-Course Scores on The UT Austin Test for Credit in Japanese in 
Relation to Perlonnance in Japanese 41 2K: Combined Frequency 
Distnbutions and Descriptive Statistics 
Fall, 1992 
(N = 92) 





Rnal Grades m Japanese 41 2K 




1 6SI 


Unsatisfactory 


oauSTaciory 


TntPl 


oCOrGS 


(0. 1) 


/O >l \ 


N 






7 


7 


DO 




1 


1 


o4 




4 

1 


1 

1 


DO 




o 


p 


CO 




3 


O 

o 


b 1 






n 


bU 




3 


o 
o 


CO 

by 




2 


o 
c 


CO 

DO 




2 


o 
c. 


57 




4 


A 


5b 




4 


A 
H 


55 




2 


0 
c. 


54 




4 


A 


5o 




4 




52 


2 


2 


4 


51 




6 


6 


50 




3 


3 


49 


2 


4 


b 


48 






0 


47 


1 


3 


4 


46 




3 


O 

o 


45 




3 


3 


44 


2 


1 


3 


><o 




3 


Q 

o 


42 




1 


1 


41 


1 


A 

\ 


2 


*fU oo 


12 


3 




Total 


20 


72 


92 


% 


22% 


78% 


100% 


Mean Score 


40.6 


53.7 


50.8 


Standard Deviation 


6.26 


8.19 


9.51 



Mean 


Standard 


Grade 


Deviation 


2.65 


1.33 



Coefficient of 
Correlation 
r = .6621 



ERIC 



15 



Table 3.3 

Scores on the UT Austin Test for Creoit in Japanese in Relation to 
Student Performance in Japanese 41 2K: Possible Decision Scores 
and Corresponoing Accuracies of Placement-Fall. 1992 
- 92) 



ERLC 





Cumulative Numoer 


Percent of Students in 


Overall Accuracy 


Place- 


of Students 


Each Placement Category 




of Placement 


ment 


Unsatisf ^^ory 


Satisfactory 






I 


Placement 


Numoer of 


% of 


Cateaofv 


0-1 fN « 20) 


2-4 (H - 72) 


UnsaiiSfactorv 


Satisfactorv 


Accuracy 


Students 


Students 


53 - up 


Too High 0 


39 Correct 


Too High 


C% 


547o Correct 


Too High 


0 


0% 












Correct 


59 


64% 


Below 53 


Correct 20 


33 Too Low 


Correct 100% 


46% Too Low 


Too Low 


33 


36% 


52 • up 


Too High 2 


41 Correct 


Too High 


1 0% 


57% Correct 


Too High 


2 


2% 














Correct 


59 


64% 


Below 52 


Correct 18 


31 Too Low 


Correct 


90% 


43% Too Low 


Too Low 


31 


34% 


51 • up 


Too High 2 


47 Correct 


Too High 


10% 


65% Correct 


Too High 


2 


2% 












Correct 


65 


71% 


Below 51 


Correct 18 


25 Too Low 


Correct 


90% 


35% Too Low 


Too Low 


25 


27% 


50 - up 


Too High 2 


50 Correct 


Too High 


10% 


69% Correct 


Too High 


2 


2% 












Correct 


68 


74% 


Below 50 


Correct 18 


22 Too Low 


Correct 


90% 


31% Too Low 


Too Low 


22 


24% 


49 - up 


Too High 4 


54 Correct 


Too High 


20% 


75% Correct 


Too High 


4 


4% 














Correct 


70 


76% 


Below 49 


Correct 16 


1 8 Too Low 


Correct 


80% 


25% Too Low 


Too Low 


18 


20% 


48 - up 


Too High 4 


54 Correct 


Too High 


20% 


75% Correct 


Too High 


4 


4% 












Correct 


70 


76% 


Below 48 


Correct 16 


18 Too Low 


Correct 


80% 


25% Too Low 


Too Low 


18 


20% 


47 - up 


Too High 5 


57 Correct 


Too High 


25% 


79% Correct 


Too High 


5 


5% 














Correct 


72 


78% 


Below 47 


Correct 15 


1 5 Too Low 


Correct 


75% 


21% Too Low 


Too Low 


15 


16% 


46 - up 


Too High 5 


60 Correct 


Too High 


25% 


83% Correct 


Too High 


5 


5% 














Correct 


75 


82% 


Below 46 


Correct 15 


12 Too Low 


Correct 


75% 


17% Too Low 


Too Low 


12 


13% 


45 - up 


Too High 5 


63 Correct 


Too High 


25% 


88% Correct 


Too High 


5 


5% 














Correct 


78 


85% 


Below 45 


Correct 15 


9 Too Low 


Corf'ect 


75% 


13% Too Low 


Too Low 


9 


10% 


44 - up 


Too High 7 


64 Correct 


Too High 


35% 


89% Correct 


Too High 


7 


8% 














Correct 


77 


84% 


Below 44 


Correct 13 


8 Too Low 


Correct 


65% 


11% Too Low 


Too Low 


8 


9% 


43 - up 


Too High 7 


67 Correct 


Too High 


35% 


» 93% Correct 


Too High 


7 


8% 














Correct 


80 


87% 


Below 43 


Correct 13 


5 Too Low 


Correct 


S5% 


7% Too Low 


Too Low 


5 


5% 


42 - up 


Too High 7 


68 Correct 


Too High 


35«/i 


> 94% Correct 


Too High 


7 


8% 














Correct 


81 


88% 


Below 42 


Correct 13 


4 Too Low 


Correct 


657< 


> 6% Too Low 


Too Low 


4 


4% 


41 - up 


Too High 8 


69 Correct 


Too High 


40r 


> 96% Correct 


Too High 


8 


9% 














Correct 


81 


88% 


Below 41 


Correct 12 


3 Too Low 


Correct 


60*y 


0 4% Too Low 


Too Low 


3 


3% 



16 



Table 3.4 

Scores on The UT Austin Test for Credit In Japanese in Relation 
to Student Perlomriance in Japanese 41 2K: Scores Suggested 
by Six Guidelines tor Use in Selecting Decision Scores 
Fall, 1992 
(N = 92) 



Guideline Score 



1 . Expected Score for students whose performance in course was just minimally 
satisfactory (i.e., students with preliminary grades of C; see Expected Score 

row at bottom of Table 3.1). 48 

2. Score for which Expected Grade was just minimally satisfactory (i.e., C; see 44 
Expected Grade column in Table 3.1). 

3. Score for which percents of errors of students in each academic performance 
category (Unsatisfactory, Satisfactor/) were most nearly equal. (See % Too High 

and % Too Low values in middle columns of Table 3.3.) 47 

4. Score for which overall percents of errors were most nearly equal. (See % Too 

High and % Too Low values in last column of Table 3.3.) 44 

5. Score that would have cut off (or held back) approximately the same number of 
students as were in the Unsatisfactory performance group. (See Table 3.2 for the 
number of students in the Unsatisfactory group and the test score that nrwst 

nearly identifies that number of low-scohng students.) 44 

6. Score that would have maximized overall accuracy of placement. (See number 

Correct in next-to-last column of Table 3.3.) 42 



o 19 

ERIC 
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ERIC 



Table 4.1 

End-of-Course Scores on The UT Austin Test for Credft in Japanese in Relation to Pertorniance in 
Japanese 41 2L: Frequency Distnbutions, Descnptrve Statistics. Regression Equations. 
Expected Grades, and Expected Scores-Spnng, 1992 
N = 58 



Final Grades in Jaoanese 412L 



Test 


Expected 


0 


1 


2 


3 


4 


Total 


Scores 


Grades 


F 


D 




a 




N 


73-79 


3.92-4.00 


1 




1 


5 


5 


72 


3.86 


! 




1 


2 


2 


71 


3.80 


1 








1 


70 


3.74 


_ ^ 1 




i 1 


1 


69 


3.68 










1 


1 


68 


3.62 




\ 


1 


1 


2 


67 


3.56 










1 


1 


66 


3.51 










2 


2 


65 


3.45 












0 


64 


3.39 








1 




1 


63 


3.33 










2 


2 


62 


3.27 








1 




1 


61 


3.21 












0 


60 


3.15 








1 




1 


59 


3.09 






1 


2 




3 


58 


3.03 








2 




2 


57 


2.97 






1 




1 


2 


56 


2.91 






1 


1 




2 


55 


2.86 






1 


4 




5 


54 


2.80 






1 


1 




2 


53 


2.74 






1 


1 




2 


52 


2.68 








4 




4 


51 


2.62 








1 




1 


50 


2.56 






1 






1 


49 


2.50 




1 


1 


1 




3 


48 


2.44 












0 


47 


2.38 






1 


1 




2 


46 


2.32 








2 




2 


45 


2.26 


1 






2 




3 


44 


2.21 






1 






1 


43 


2.15 






1 






1 


42 


2.09 












0 


41 


2.03 






1 






1 


40 


1.97 












0 


39 


1.91 












0 


38 


1.85 












0 


37 


1.79 








1 




1 


Total 


1 


1 


12 


27 


17 


58 


% 






21% 


47% 


29% 


100% 


Mean Score 


45.0 


49.0 


50.7 


53.5 


69.7 


57.4 


Standard Deviation 


0.00 


0.00 


5.68 


6.55 


5.57 


10.02 


Expected Score 


33 


41 


49 


57 


66 





Expected Grade » 

(Test Score x 
0.05908) ' 0.3942 



Expected Score « 
(Preliminary Grade X 
8.1905) +32.8768 



Mean 


Standard 


Grade 


Deviation 


3.00 


0.85 



Coetticient of 
Correlation 
r - .6956 
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Table 4.2 

End-of-Course Scores on The UT Austin Test for Credit in Japanese in 
Relation to Pedormance in Japanese 41 2L: Combined Frequency 
Distnbutions and Descriptive Statistics-Spring. 1992 
(N = 58) 



1 

1 


Final Graces in Japanese 412L 




Test 


Unsatistactory 


Satistactory 


Total 


Scores 


fO. 1) 


f2'41 


N 


73-79 1 


1 


5 1 


5 


72 1 


r 


2 


2 


71 i 




1 


1 


Zo L 




1 


1 


69 1 




1 


68 


\ — i r 


2 


67 1 


( 1 


1 


66 ! 




2 


2 


65 






0 


64 




1 


1 


63 




2 


2 


62 






1 


61 




1 


0 


60 




1 


1 


59 




3 


3 


58 




2 


2 


57 




2 


2 


56 




2 


2 


55 




5 


5 


54 




2 


2 


53 




2 


2 


52 




4 


4 


51 




1 


1 


50 




1 


1 


49 


1 


2 


3 


48 






0 


47 




2 


2 


46 




2 


2 


45 


1 


2 


3 


44 




1 


1 


43 




1 


1 


42 






0 


41 




1 


1 


40 






0 


39 






0 


38 






0 


37 




1 


1 


Total 


2 


56 


58 


% 


3% 


97% 


100% 


Mean Score 


47 0 


57.8 


57.4 


Standard Deviation 


2.00 


9.99 


10.02 
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Table 4.3 

Scores on the UT Austin Test for Credit in Japanese m Relation to 
Student Perlormance in Japanese 41 2L: Possible Decision Scores 
and CorresDonaing Accuracies of Placement-Spring, 1992 
fN » 58^ 



ERLC 



Place- 
ment 

Cateaorv 


Cumulative Numoer 
of Students 


Percent of Students in 
Each Placement Category 


Overall Accuracy 
of Placement 


Unsatisfactory 

\J'] [Pi m ii) 


Satisfactory 

O A /M CS\ 
^•4 ifN 3 DD) 


UnsatJstactorv 


Satisfactorv 


Placement 
Accuracy 


Number ot 
Students 


%of 
Students 


53 - up 
Below 53 


Too High C 
Correct 2 


38 Correct 
1 o Too Low 


Too Hign 0% 
Correct 1 00% 


68% Correct 
22% Too Low 


Too High 
Correct 
Too Low 


0 
40 
18 


0% 
69% 
31% 


52 - up 
Below 52 


Too High 0 
uorreci ^ 


42 Correct 
1 4 Too Low 


Too High 0% 
uorrect 1 00% 


75% Correct 
25% Too Low 


Too High 
Correct 
Too Low 


0 
44 
14 


0% 
76% 
24% 


51 - up 
Below 51 


Too High 0 
Correct 2 


43 Correct 
1 J 100 Low 


Too High 0% 
Correct 1 00% 


77% Correa 
23% Too Low 


Too High 
Correct 
Too Low 


0 
45 
13 


0% 
78% 
22% 


50 - up 
Below 50 


Too High 0 
Correct 2 


44 Correct 
1 2 Too Low 


Too High 0% 
Correct 1 00% 


79% Correct 
21% Too Low 


Too High 
Correct 
Too Low 


0 
46 
12 


0% 
79% 
21% 


49 - up 
Below 49 


Too High 1 
Correct 1 


46 Correct 
10 Too Low 


Too High 50% 
Correct 50% 


82% Correct 
18% Too Low 


Too High 
Correct 
Too Low 


1 

47 
10 


2% 
81% 
17% 


48 - up 
Below 48 


Too High 1 
Correct 1 


46 Correct 
1 0 Too Low 


Too High 50% 
Correct 50% 


82% Correct 
18% Too Low 


Too High 
Correct 
Too Low 


1 

47 
10 


2% 
81% 
17% 


47 - up 
Below 47 


Too High 1 


48 Correct 

O 1 00 LOW 


Too High 50% 
'^orreci dU/o 


86% Correct 
14% Too Low 


Too High 
Correct 
Too Low 


1 

49 
8 


2% 
84% 
14% 


46 • up 
Below 46 


Too High 1 

CnrrftCt 1 


50 Correct 

O J UU LOW 


Too High 50% 
'^orreci ouvo 


89% Correct 
11% Too Low 


Too High 
Correct 
Too Low 


1 

51 
6 


2% 
88% 
10% 


45 - up 
Below 45 


Too High 2 
Correct 0 


52 Correct 
4 Too Low 


Too High 100% 
Correct 0% 


93% Correct 

Tnn \ r\\n 
1 *o I UU LOW 


Too High 
Correct 
Too Low 


2 
52 
4 


3% 
90% 
7% 


44 • up 
Below 44 


Too High 2 
Correct 0 


53 Correct 
3 Too Low 


Too High 100% 
Correct 0% 


95% Correct 
5% Too Low 


Too High 
Correct 
Too Low 


2 
53 
3 


3% 
91% 
5% 


41 - up 
Below 41 


Too High 2 
Correct 0 


55 Correct 
1 Too Low 


Too High 100% 
Correct 0% 


98% Correct 
2% Too Low 


Too High 
Correct 
Too Low 


2 
55 
1 


3% 
95% 
2% 


39 - up 
Below 39 


Too High 2 
Correct 0 


55 Correct 
1 Too Low 


Too High 100% 
Correct 0% 


98% Correct 
2% Too Low 


Too High 
Correct 
Too Low 


2 
55 
1 


3% 
95% 
2% 


37 - up 

Below 37 


Too High 2 
Correct 0 


56 Correct 
0 Too Low 


Too High 100% 
Correct 0% 


100% Correct 
0% Too Low 


Too High 
Correct 
Too Low 


2 
56 

0 


3% . 
97% 
0% 



22 



BEST COPY AVAILABLE 



Table 4.4 

Scores on The UT Austin Test tor Credit In Japanese in Relation 
to Student Performance in Japanese 41 2L: Scores Suggested 
by Six Guidelines for Use in Selecting Decision Scores 
Spnng, 1992 
(N = 58) 



Guideline 



Score 



1 . Expected Score for students whose performance m course was just minimally 
satisfactory (i.e., students with preliminary grades of C; see Expected Score 
row at bottom of Table 4.1). 

2. Score for which Expected Grade was just minimally satisfactory (i.e., C; see 
Expected Grade column in Table 4.1). 



49 
41 



3. Score for which percents of errors of students in each academic performance 
category (Unsatisfactory, Satisfactory) were most nearly equal. (See % Too High 
and % Too Low values in middle columns of Table 4.3.) 



50 



4. Score for which overall percents of errors were most nearly equal. (See % Too 

High and % Too Low values in last column of Table 4.3.) 43 

5. Score that would have cut off (or held back) approximately the same nunnber of 
students as were in the Unsatisfactory performance group. (See Table 4.2 for the 
number of students in the Unsatisfactory group and the test score that most 

nearly identifies that number of low-scoring students.) 43 



6. Score that would have maximized overall accuracy of placement. (See number 
Con-ect in next-to-last column of Table 4.3.) 



37 
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these courses the distribution of grades was skewed towards the higher grades; that is, more 
people received the grades of A and B than tlie grades of h\ IX or C Second, ov erall mean test 
score increased from the sequence of courses Japanese 506 to Japanese 4121,. This is to be 
expected. On average, a student in Japanese 412K, say, should perform better than students in 
Japanese 506 and Japanese 507 but not as well as students in Japanese 41 2L. Third, within any 
course, mean score increased as preliminary tlnal grade increased. For example, students who 
earned the preliminary grade of /\ had, on average, a higher test score than did all other students. 
Students who eamed a grade of B had a higher average score than did all students except the A 
students, and so on. In addition, for any preliminary grade, mean test scores increased as the 
course level increased. Students in Japanese 507 who received the preliminary grade of C had a 
higher mean test score than C students in Japanese 506. but a lower mean test score than students 
in both Japanese 41 2K and 412L. These two findings are congruent with what one would expect, 
namely, that the better performing students, both within a course and across courses in a sequence, 
have a higher mean test score than do less well ..pertbrming students. Finally, expected score 
increases in a fashion similar to the mean test scores. This is to be expected given the distribution 
of grades in the four courses. 



Recommendations and Decisions 

After analyzing the data and meeting with the instructors of Japanese courses, a set of cut 
scores was determined for credit in each course. These scores allow students to cam credit in each 
of the four lower division Japanese courses with a grade of A, B, or C. Students may, if they 
desire, request credit only, rather than a letter grade, provided tliat they have attained a cut score 
that would allow them to petition for credit. The scores decided on are presented in Figure 1 . 



2-^ 

ERIC 



Score 506 507 412K 412L Score 




Figure 1. Credit awarded at UT Austin in lower division Japanese courses for various scores 
on the Placement Test. 
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